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FOREWORD 


This  report  is  an  outgrowth  of  the  work  of  the 
Defense  Science  Board  Task  Force  on  Test  and  Evaluation, 
and  the  checklists  herein  have  been  derived  from  the 
study  of  past  major  weapon  system  programs. 

The  T&E  expert  in  reading  this  volume  will  find 
many  precepts  which  will  strike  him  as  being  too  obvious 
to  be  included  in  checklists  of  this  type.  These  items 
are  included  because  examples  were  found  where  even  the 
obvious  has  been  neglected,  not  because  of  incompetence 
or  lack  of  personal  dedication  by  the  people  in  charge 
of  the  program,  but  because  of  financial  and  temporal 
pressures  which  forced  competent  managers  to  compromise 
on  their  principles.  It  is  hoped  that  the  inclusion  of 
the  obvious  will  prevent  repetition  of  the  serious  errors 
which  have  been  made  in  the  past  when  such  political, 
economic  and  temporal  pressures  have  forced  project 
managers  to  depart  from  the  rules  of  sound  engineering 
practices. 

In  the  long  run,  taking  short  cuts  during  T&E  to 
save  time  and  money  will  result  in  significant  increases 
in  the  overall  costs  of  the  programs  and  in  the  delay  of 
the  delivery  of  the  corresponding  weapon  systems  to  the 
combatant  forces. 
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T&E  GUIDELINES  FOR  GROUND  VEHICLE  SYSTEMS 

) 

The  checklist  items  presented  here  are  specifically  applicable  to 
ground  vehicle  testing  and  evaluation.  It  is  suggested  that  the  user  of 
this  volume  also  refer  to  the  Report  of  the  Defense  Science  Board  on  Test 
and  Evaluation  which  contains  general  checklist  items  also  applicable  to 
this  system  T&E  program.  The  checklist  items  presented  here  are  organized 
into  time  phases  of  the  acquisition  process  oriented  to  the  DSARC  cycle. 

The  checklists  cover  various  aspects  of  the  major  activities  that 
should  be  underway  during  a  given  time  period.  Hence,  a  checklist  might 
cover  the  (1)  evaluation  of  work  that  occurred  in  the  previous  phase, 

(2)  conduct  of  tests  planned  in  the  previous  phase  and  executed  in  the 
subject  phase,  and  (3)  plans  and  other  preparatory  actions  for  test  sche¬ 
dules  to  be  conducted  in  a  subsequent  phase.  For  reasons  such  as  this, 
items  on  some  subjects,  such  as  development  test  plans,  may  appear  in  more 
than  one  phase.  In  addition,  since  the  Services  and  the  DSARC  have  flexi¬ 
bility  in  deciding  how  rapidly  to  progress  in  the  Validation  Phase,  there 
may  be  cases  where  the  Request  for  Proposals  (RFPs),  proposal  evaluations, 
source  selections,  or  contract  negotiations  may  occur  after  the  DSARC 
approves  full-scale  development  Instead  of  before.  For  this  reason,  it 
is  recommended  that  previous  checklists  in  the  Validation  Phase  be  re¬ 
viewed  when  entering  the  Full-Scale  Engineering  Development  Phase.  The 
following  are  the  phases  used  in  this  report. 

CONCEPTUAL  PHASE 

The  checklist  items  in  this  phase  are  for  guidance  in  evaluating  T&E 
activities  during  the  Conceptual  Phase  of  the  acquisition  of  the  system. 
This  phase  (often  research  and  exploratory  development)  precedes  the  first 
DSARC  milestone  and  is  focused  on  the  development  of  a  weapon  system  con¬ 
cept  that  offers  high  prospects  of  satisfying  an  identified  military  need. 

Although  not  called  for  in  DoD  Directive  5000.1  specifically,  the 
objectives  of  this  phase  should  be: 
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1.  To  verify  that  there  is  a  military  need  for  the  proposed 
system. 

2.  To  demonstrate  that  there  is  a  sound  physical  basis  for  a  new 
weapon  system. 

3.  To  formulate  a  concept,  based  on  demonstrated  physical 
phenomena,  for  satisfying  the  military  need. 

4.  To  show  that  the  proposed  solution  is  superior  to  its  com¬ 
petitors  in  terms  of  potential  effectiveness,  probability  of 
success,  probable  cost,  impact  on  the  U.S.  military  posture, 
and  development  risks. 

5.  To  analyze  the  technology  outlook  and  the  military  need  to 
show  that  it  is  better  to  start  advanced  development  now 
rather  than  to  wait  for  future  technological  Improvements. 

6.  To  Identify  the  key  risk  areas  and  critical  issues  that  need 
to  be  resolved  before  full-scale  development  is  initiated. 

The  most  important  product  of  this  phase  is  the  Development  Concept 
Paper  (DCP)  or  its  equivalent.  The  DCP  defines  program  issues,  including 
special  logistics  problems,  program  objectives,  program  plans,  performance 
parameters,  areas  of  major  risk,  system  alternatives,  and  acquisition 
strategy. 

VALIDATION  PHASE 

The  checklist  items  in  this  phase  are  for  guidance  in  conducting  T&E 
during  the  Validation  Phase  (the  time  between  when  the  DSARC  recommends 
approval  of  the  DCP  for  the  first  time  and  when  the  DSARC  recommends  full- 
scale  development  of  the  system). 

Nhile  these  objectives  are  not  spelled  out  in  the  DoD  Directive  5000.1, 
the  objectives  of  the  Validation  Phase  should  be  to  confirm: 

1.  The  need  for  the  selected  system  in  consideration  of  the  threat, 
system  alternatives,  special  logistics  needs,  estimates  of 
development  costs,  preliminary  estimates  of  life  cycle  costs 
and  potential  benefits  in  context  with  overall  DoD  strategy  and 
fiscal  guidance. 

2.  The  validity  of  the  operational  concept. 

3.  That  development  risks  have  been  identified  and  solutions  arc 
in  hand. 

4.  Realism  of  the  plan  for  full-scale  development. 
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In  the  pursuit  of  the  above  objectives,  it  is  likely  that  advanced 
development  T&G  will  be  conducted  to  resolve  issues.  In  some  cases,  an 
RFP  for  full-scale  engineering  development  will  be  prepared,  proposals 
will  be  received  and  evaluated,  and  contracts  negotiated  in  preparation 
for  seeking  DSARC  approval  for  the  next  phase.  Therefore,  some  checklist 
items  are  included  to  help  ensure  that  this  work  properly  reflects  the 
T&E  Interests  in  this  and  subsequent  phases.  For  example,  the  RFP  must 
include  adequate  guidance  to  ensure  that  sufficient  resources  and  time  are 
available  so  that  engineering  effort  can  properly  support  the  initial  DT&E 
with  hardware,  software,  technical  data,  and  training. 

The  primary  emphasis  of  OSD/T&E  activities  is  with  items  3  and  4  above. 
Special  attention  should  be  given  to  the  planning  of  IOT&E  activity  as  it 
is  incorporated  in  the  engineering  development  contract  as  well  as  the 
DT&E  associated  with  addressing  the  critical  issues  and  areas  of  major 
risk  identified  in  the  DCP. 

FULL-SCALE  ENGINEERING  DEVELOPMENT  PHASE 

The  checklist  items  contained  in  this  phase  are  for  guidance  in 
conducting  T&E  during  the  Full-Scale  Engineering  Development  Phase.  This 
Includes  the  major  DT&E  and  the  IOT&E  conducted  prior  to  the  major  pro¬ 
duction  decision.  By  this  time,  the  system  is  well  defined  and  is 
becoming  a  unique  item  and,  hence,  sound  judgment  must  be  applied  in  using 
these  checklist  items. 

To  enter  the  Engineering  Development  Phase,  the  DSARC  will  have: 

•  Confirmed  the  need  in  consideration  of  the  threat,  alternatives, 
logistic  needs,  cost,  and  benefits. 

•  Identified  development  risks. 

•  Confirmed  the  realism  of  the  development  plan. 

Given  the  above,  the  primary  objectives  of  the  DT&E  should  be  to: 

1.  Demonstrate  that  the  engineering  and  design  and  development 
process  is  complete  and  that  the  design  risks  have  been  mini¬ 
mized  (the  system  is  ready  for  production). 

2.  Demonstrate  that  the  system  will  meet  specifications. 
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The  primary  objectives  of  the  IOT&E  should  be  to: 

3.  Assess  operational  suitability  and  effectiveness. 

4.  Validate  organizational  and  employment  concepts. 

5.  Determine  training  and  logistic  requirements. 

In  addition,  the  validity  of  the  plan  for  the  remainder  of  the  program 
must  be  confirmed  by  the  DSARC  before  substantial  production/deployment 
will  be  recommended  to  the  Secretary  of  Defense. 

The  level  of  OSD/T&E  activity  is  highest  during  this  phase.  The 
IOT&E  plan  must  be  designed,  the  tests  conducted,  and  the  data  analyzed 
to  evaluate  the  inputs  associated  with  the  primary  objectives.  These 
tests  should  not  be  conducted  until  the  primary  objectives  of  the  DT&E 
have  been  met.  Thus,  OSD/T&E  activity  is  required  to  assess  that  the  DT&E 
major  milestone — the  system  is  ready  for  production — has  been  achieved. 
Close  monitoring  of  the  T&E  Service  activity  is  required  during  the  latter 
stages  of  this  phase. 

SUBSTANTIAL  PRODUCTION/DEPLOYMENT  PHASE 

The  checklist  items  contained  in  this  phase  are  for  guidance  in  con¬ 
ducting  T&E  after  the  substantial  production  decision  has  been  made  by  the 
DSARC.  This  Includes  DT&E  and  follow-on  OT&E  to  be  conducted  on  the  early 
production  items. 

To  enter  the  Production/Deployment  Phase,  the  DSARC  will  have  re¬ 
viewed  the  program  to  confirm: 

•  The  need  for  the  system. 

•  A  practical  engineering  design  with  adequate  consideration  of 
production  and  logistic  problems  is  complete. 

•  All  technical  uncertainties  have  been  resolved  and  opera¬ 
tional  suitability  has  been  determined  by  T&E. 

•  The  realism  of  the  ptlan. 

The  primary  objective  of  the  DT&E  in  this  phase  should  be  to: 

1.  Verify  that  the  production  system  meets  specifications. 

The  primary  objectives  of  the  follow-on  OT&E  should  be  to: 
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2.  Validate  the  operational  suitability  and  effectiveness. 

3.  Optimize  organization  and  doctrine. 

4.  Validate  training  and  logistic  requirements. 

At  this  point,  the  OSD/T&E  activity  is  similar  to  that  in  the 
previous  phase;  however,  much  of  the  testing  is  verification  that  the 
production  system  performance  is  as  expected.  Hence,  most  of  the  items 
in  the  previous  phase  are  appropriate  to  this  phase,  especially  those 
related  to  OT&E. 


I .  CONCEPTUAL  PHASE 


The  prime  objective  of  the  Conceptual  Phase  of  a  ground  vehicle  program 
is  to  verify  the  feasibility  of  the  new  concept.  The  genesis  of  the 
program  is  usually  built  on  a  growing  threat  or  a  new  need  where  existing 
equipments  have  serious  deficiencies,  modification  costs  are  excessive,  and 
a  more  efficient  or  capable  new  system  is  proposed.  The  new  system  is 
usually  based  on  studies,  IR&D,  exploratory  or  advanced  development  and 
technology  indications  During  the  concept  formulation  phase,  a  DCP  is 
normally  proposed  which  develops  a  solution  by  presenting  such  matters  as 
justification,  a  program  plan,  schedules,  tradeoffs,  costs,  alternatives, 
risks,  and  issues. 

This  phase  requires  testing  which  will  demonstrate  the  capability  of 
the  concept  system  or  vehicle  to  meet  the  objectives  and  performance 
requirements  which  have  been  established.  A  preliminary  test  plan  will  be 
designed  for  use  during  the  Validation  Phase.  Experience  gained  from 
problems  encountered  in  previous  programs  should  be  applied  at  this  point. 
The  T&E  checklist  for  this  phase,  portions  of  which  may  have  continuing 
application  in  subsequent  phases,  includes: 

1.  Preparation  of  Test  Plans 

2.  Validation  Test  Plans 

3.  Performance  Characteristics  Range 

4.  Operating  Degradation 

5.  Test  Personnel 

6.  Design  Reviews 

7.  Prototype  Vehicles 

8.  Test  Facilities  and  Scheduling 
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PREPARATION  OF  TEST  PLANS 


It  is  necessary  that  a  detailed  evaluation  criteria  be  established 

which  includes  all  Items  that  are  to  be  tested. 

Without  such  criteria  the  results  of  the  tests  may  not  provide  the 
performance  information  needed  by  the  developer  and  the  user.  In  some  in¬ 
stances  in  the  past  important  data  were  not  obtained  in  the  initial  tests 
and  retesting  was  necessary  or  decisions  were  made  in  the  absence  of  com¬ 
plete  test  data. 

2.  VALIDATION  TEST  PLANS 

Prior  to  DSARC  I,  a  plan  should  be  prepared  for  an  evaluation  of  the 

overall  T&E  program. 

As  part  of  this,  a  detailed  test  and  evaluation  plan  for  those  tests 
to  be  conducted  prior  to  DSARC  II  to  validate  the  concept  and  hardware 
approach  to  the  vehicle  system  should  be  developed.  The  objective  of  the 
validation  test  plan  is  to  fully  evaluate  the  performance  characteristics 
of  the  new  concept  vehicle.  This  test  plan  cannot  be  developed,  of  course, 
until  the  performance  characteristics  are  defined.  Even  at  this  early 
phase  some  effort  should  be  planned  to  evaluate  reliability.  Plans  to 
evaluate  human  factors,  serviceability  and  maintenance  should  be  included. 

This  teot  plan  should  include: 

•  Understandable  description  of  the  characteristics. 

•  How  the  characteristics  will  be  evaluated. 

•  Facilities  needed  to  make  evaluations. 

•  Schedule  of  evaluations  and  facilities. 

•  Reporting  procedure. 

•  Cost  estimates. 

Detailed  evaluation  criteria  (e.g.,  a  ROC  supported  by  any  additional 
necessary  documentation)  should  be  available  prior  to  the  start  of  testing 
of  a  system.  Where  practical,  detailed  quantitative  goals  are  desirable 
(e.g.,  a  tank  shall  have  a  minimum  top  speed  of  35  mph  and  a  minimum  draw 
bar  pull  of  25,000  pounds  on  hard  roads).  Duplication  and  redundance  of 
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goals  should  be  minimized  (e.g.,  "truck  should  have  a  minimum  fuel  capacity 
of  50  gallons,  minimum  hard  road  efficiency  of  6  miles  per  gallon  and  mini¬ 
mum  hard  road  range  of  300  miles"  is  redundant.  Only  two  of  the  specifica¬ 
tions  are  needed  [operationally,  only  the  range  specification  is  meaning¬ 
ful]). 

The  interaction  of  contract  requirements  and  test  plans  should  be 
recognized.  For  instance,  the  contract  incentive  fee  criteria  may  con¬ 
strain  the  developer  from  exploring  the  full  performance  envelope,  because 
of  fear  that  failure  will  reduce  payments  or  jeopardize  continuation  of  the 
program,  rather  than  encourage  the  developer  to  determine  the  full  perfor¬ 
mance  envelope  at  the  earliest  possible  date. 

For  example,  a  vehicle  with  a  specification  requirement  to  travel  at 
35  mph  (and  which  has  satisfactorily  demonstrated  this  capability)  should  be 
tested  to  determine  its  maximum  achievable  speed  without  any  penalty  to  the 
developer  if  failures  (e.g.,  of  the  engine)  occur  at  the  higher  speed.  Es¬ 
pecially  do  not  allow  test  criteria  to  be  limited  to  a  single  evaluation  cri¬ 
terion  such  as  hit  or  miss  of  a  target. 

3.  PERFORMANCE  CHARACTERISTICS  RANGE 

Stated  performance  characteristics  derived  from  studies  should  be 

measured  early  in  the  program. 

Unrealistic  performance  requirements  can  lead  to  false  starts  and 
costly  delays.  In  some  past  programs  the  mismatch  of  the  performance 
called  for  in  the  requirement  with  that  technically  achievable  has  been  the 
major  cause  of  serious  problems  and  delays  in  development  programs.  Per¬ 
formance  characteristics  specified  on  the  basis  of  analytical  and  empirical 
studies  conducted  prior  to  DSARC  I,  in  which  the  range  of  critical  perfor¬ 
mance  characteristics  has  been  determined,  should  be  measurable  through 
bench  and  laboratory  (which  includes  proving  ground  )  testing.  The  test 
design  and  the  number  of  tests  should  be  adequate  to  provide  results  with 
confidence  limits  compatible  with  the  statements  of  desired  characteristics. 
For  example,  a  demonstration  that  a  20  percent  increase  in  suspension  travel 
will  permit  a  10  percent  increase  in  speed  over  a  standard  obstacle  without 
increasing  the  g's  experienced  by  the  operator  can  be  made  with  a  very  small 
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number  of  tests.  Testing  in  advanced  development  should  be  planned  to 
explore  the  performance  characteristics  over  a  broad  range  so  as  to  provide 
insight  into  system  performance  over  the  expected  range  of  values  and  not 
just  at  a  single  point.  For  example,  the  envelopment  of  a  new  type  of  armor 
plate  would  include  firings  at  sample  plate  segments  using  a  variety  of 
types  of  projectile  (kinetic  energy,  shaped  charge)  at  various  impact 
velocities  and  angles  of  incidence. 

4.  OPERATING  DEGRADATION 

System  performance  degrades  under  field  conditions.  Anticipated  de¬ 
gradation  must  be  considered  during  test  and  evaluation . 

When  a  system  must  operate  at  peak  performance  during  DT/OT  to  meet 
the  specified  requirements  it  then  will  likely  perform  at  a  lesser  level 
when  operated  in  the  field.  In  several  past  programs  vehicular  systems 
were  beefed  up  and  stressed  to  meet  DT/OT  test  conditions.  When  final’y 
fielded,  the  performance  was  below  the  minimum  needed. 

The  system  concept  and  possible  implementation  must  not  hinge  on  the 
requirement  for  the  vehicle  system  or  subsystems  to  be  finely  tuned  when  the 
expected  operational  environment  suggests  that  this  will  not  be  likely.  The 
vehicle  system  should  not  degrade  significantly  as  a  result  of  detuning 
caused  from  expected  operational  usage.  For  example,  if  the  cross-country 
mobility  capability  is  expected  to  degrade  with  operational  use*  (and  the 
ability  to  perform  typical  missions  depends  on  this  capability),  then  tests 
of  the  mobility  capability  should  be  conducted  with  the  vehicle  degraded  to 
operational  levels  to  establish  the  sensitivity  to  this  factor. 


♦Causes  might  include  loss  of  engine  power  due  to  normal  engine  wear,  dirt 
in  engine  air  filter  and  higher  eng’ne  operating  temperatures  caused  by 
accumulated  deposits  of  dirt,  grease,  etc.,  loss  of  ground  traction  due  to 
tread  or  track  wear,  loss  of  vehicle  ride  comfort  due  to  normal  wear  of  sus¬ 
pension  components.  In  addition,  performance  losses  may  also  be  caused  by 
environmental  variations  including  variations  in  air  temperature,  humidity, 
ground  moisture,  etc. 
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5. 


TEST  PERSONNEL 


The  test  director  and/or  key  members  of  the  test  planning  group  within 

the  project  ot'fice  should  have  significant  T&E  experience. 

If  the  requisite  experience  does  not  exist  at  the  appropriate  levels 
within  the  project  office,  test  plans  may  be  based  on  too  shallow  or  too 
naive  a  conception  of  the  role  and  potential  utility  of  the  T&E  process. 

In  a  number  of  past  programs  key  test  personnel  were  assigned  to  T&E  slots 
with  little  prior  exposure  to  T&E  or  its  management,  and  with  inadequately 
experienced  support  as  well.  The  test  planning  group  should  have  personnel 
experienced  in  engineering  testing,  development  testing  and  operational 
testing.  Operational  experience  is  also  desirable  (a  test  planner  on  a 
tank  program  should  have  armor,  rather  than  helicopter,  experience).  This 
experience  should  be  available  early  in  this  phase  and  all  efforts  should 
be  made  to  encourage  these  people  to  remain  with  the  system  project  office 
through  the  T&E  phases  of  the  program. 

6.  DESIGN  REVIEWS 

T&E  factors  and  experience  must  influence  the  system  design. 

The  application  of  knowledge  derived  from  past  experience  can  be  a 
major  asset  in  arriving  at  a  sound  system  design.  In  a  few  instances  de¬ 
sirable  design  features  were  overlooked  by  not  capitalizing  on  lessons 
learned  in  previous  programs. 

The  DT&E  program  initially  will  progress  through  a  design  stage.  It 
is  during  this  period  that  the  detailed  design  will  be  put  on  paper  and 
possibly  mocked-up.  At  the  formal  design  reviews,  the  T&E  people  and  all 
interested  parties  will  have  an  opportunity  to  view  the  hardware  plans  and 
to  make  recommendations.  T&E  related  matters  should  be  considered  at  these 
design  reviews  because  changes  are  difficult  to  make  later.  For  example, 
such  factors  as  the  layout  of  a  tank  turret,  minimization  of  the  number  of 
connectors,  convenience  of  communications  equipment  usage,  ammunition  posi¬ 
tioning  and  accessibility  for  loading,  and  similar  items  should  be  examined 
at  this  time.  In  addition,  previous  programs  should  be  examined  for  problem 
areas  to  identify  potential  areas  of  concern. 
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7. 


PROTOTYPE  VEHICLES 


When  high  technical  risk  Is  present,  development  should  be  structured 
around  the  use  of  one  or  more  prototype  vehicles  designed  to  prove 
the  system  concept  under  realistic  operational  conditions  before  pro¬ 
ceeding  tc  engineering  development. 

In  most  systems  there  are  generally  several  subsystems  which  represent 
the  major  technical  risks.  It  has  not  been  uncommon  for  systems  to  be  plac¬ 
ed  in  full-scale  development  without  adequate  proof  of  available  technology. 

It  is  frequently  desirable  to  take  a  risk;  however,  when  an  implied 
commitment  to  production  is  involved  the  technology  should  be  operationally 
proof  tested  prior  to  commencing  full-scale  development.  Avoid  the  tempta¬ 
tion  of  thinking  that  any  vehicle  system,  subsystem,  or  component  is  "state- 
of-the-art"  until  it  is  working  in  the  field.  Provide  a  sufficient  number 
of  prototypes  to  perform  the  necessary  testing  without  delays  caused  by 
unavailability  of  test  vehicles. 

8.  TEST  FACILITIES  AND  SCHEDULING 

Before  DSARC  I,  test  range  and  resource  requirements  to  conduct  valida¬ 
tion  tests  should  be  identified  along  with  a  tentative  schedule  of  test 
activities. 

If  inter-service  testing  is  contemplated,  preliminary  plans  for  such 
testing  should  be  coordinated  with  the  cooperating  service. 

The  applicability  of  the  test  ranges  and  the  adequacy  of  the  facili¬ 
ties  and  instrumentation  should  be  verified  insofar  as  possible  alternative 
approaches  (different  ranges,  etc.)  and  instrumentation  improvements  needed 
should  he  specified.  Of  prime  importance  are  the  constraints  to  be  placed 
on  the  test  because  of  the  range  and  instrumentation.  Evaluation  of  per¬ 
formance  at  125°  F  ambient  is  difficult  when  ambient  temperatures  do  not 
exceed  90°  F.  If  range  and  instrumentation  factors  are  found  to  cast  sig¬ 
nificant  doubt  on  the  meaningfulness  of  the  test  data  because  of  a  lack  of 
operational  realism,  the  steps  necessary  to  assure  meaningful  data  should 
be  identified  and  planned. 


12 


Some  systems  may  require  MCP  (Military  Construction  Program)  facili¬ 
ties  for  portions  of  the  testing.  The  long  lead  times  to  obtain  authoriza¬ 
tion,  appropriations,  and  to  construct  or  obtain  facilities  can  pace  a 
program.  For  example,  many  steps  and  considerable  time  are  involved  in 
getting  facilities  ready  and  test  gear  in  place  to  start  system  tests. 

If  the  ability  of  a  tank  to  travel  through  wooded  areas  containing  trees 
up  to  6  inches  in  diameter  is  to  be  evaluated,  a  location  must  be  obtained 
where  such  trees  may  be  knocked  down  and  considerable  damage  done  to  the 
undergrowth. 

Most  of  tne  steps  to  program,  justify,  and  construct  facilities  must 
be  done  mainly  in  series  so  that  a  long  time  span  is  involved.  The  com¬ 
pletion  of  DT&E  and  the  operational  testing  may  require  the  MCP  facility. 
These  matters  must  be  considered  in  preparing  and  evaluating  a  test  plan 
where  MCP  is  involved. 

In  general,  the  OT&E  plan  should  include  engagements  in  the  environ¬ 
ments  in  which  the  new  system  is  expected  to  operate.  Testing  may  be 
addressed  in  several  phases,  such  as  (in  the  case  of  tanks): 

(a)  One-on-one  testing  against  existing  US  tanks  and  available  simu¬ 
lators  of  the  assumed  threat. 

(b)  Multiple  tank  testing  in  a  multiple  vehicle  environment  including 
mobile  artillery,  anti-tank  weapons  and  aircraft. 

(c)  Comparative  testing  of  the  new  tank  system  with  existing  systems 
to  estimate  the  increased  capability. 
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II.  VALIDATION  PHASE 

In  the  Validation  Phase  the  issues  raised  by  the  DCP  may  be  resolved 
by  conducting  tests  of  the  system.  The  development  and  operational  plans 
will  be  defined  in  considerable  detail. 

Specific  actions  during  this  phase  are: 

•  Verify  that  the  vehicle  system  does  satisfy  the  specified  perfor¬ 
mance  requirements. 

•  Analyze  the  characteristic  behavior  of  risk  components  (those 
approaching  or  exceeding  the  limits  of  current  state-of-the-art). 

•  Refine  the  preliminary  requirements  for  performance,  reliability, 
human  factors,  serviceability,  and  maintenance  in  the  light  of 
test  results. 

•  Develop  and  refine  the  test  plan  for  the  engineering  development 
phase . 

•  Develop  design  criteria  information. 

Guidelines  from  the  previous  phases  may  continue  to  apply  and  should  be 
reviewed.  The  checklist  for  this  phase  is  subdivided  into  the  following 
categories: 

1.  Vulnerability 

2.  Gun  and  Ammunition  Performance 

3.  Increased  Complexity 

4.  Component  Interfaces 

5.  Determination  of  Test  Conditions 

6.  Test  Plan  Development 

7.  Demonstration  Tests 

8.  Reliability  Testing 

9.  Human  Factors 

10.  Test  Plan  Scheduling 

11.  Test  Failures 


Preceding  page  blank 
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1.  VULNERABILITY 

The  vulnerability  of  vehicles  should  be  estimated  on  the  basis  of 

testing. 

E.g.,  fuel  vulnerability  tests  should  be  performed  on  tactical  vehi¬ 
cles.  This  includes  flammability  of  the  fuel/fuel  tank  combination  (e.g., 
whether  or  not  fuel  cells  are  fitted)  under  various  situations  (e.g., 
kinetic  energy  penetration,  including  small  arms  fire,  shaped  charge  jet 
exposure).  Also  included  are  any  fire  extinguisher  systems,  automatic  or 
manual;  for  automatic  systems,  evaluation  ot  the  triggering  mechanism  and 
its  sensitivity  to,  e.g.,  cigarette  lighters,  cigarettes,  and  reflected 
sunlight. 

Vulnerability  tests  for  tanks  and  other  armored  vehicles  should  include 
both  plate  evaluations  (firings  ag  inst  plates  of  various  thicknesses, 
materials,  and  obliquity  angles)  and  firings  agaiast  the  complete  ballistic 
hull  and  turret.  Plate  penetration,  penetration  through  openings,  and  key¬ 
ing  of  moving  joints  should  all  be  investigated.  Analytical  studies  of  hull 
performance  (including  considerations  of  factors  such  as  angle  of  attack, 
penetration  probabilities,  effects  of  penetration,  and  type  of  kill — 
mobility  [M] ,  firepower  [F],  "K")  should  be  performed  concurrently. 

2.  GUN  AND  AMMUNITION  PERFORMANCE 

Gun  and  ammunition  development  should  be  considered  a  part  of  overall 

tank  system  development. 

When  a  new  gun  tube,  or  one  which  has  not  previously  been  mounted  on  a 
tank  chassis,  is  being  evaluated,  all  ammunition  types  (including  missiles) 
planned  for  use  in  that  system  should  be  test  fired  under  simulated  opera¬ 
tional  conditions.  If  the  gun  tube  and  ammunition  have  been  proven,  then 
any  new  ammunition  types  being  considered  should  be  test  fired  under  simu¬ 
lated  operational  conditions,  with  particular  emphasis  on  discovering  any 
potential  safety  hazards. 

Use  of  a  realistic  mockup  in  conjunction  with  hot  firings  should  per¬ 
mit  evaluation  of  shock  and  recoil  loadings  on  the  turret;  combustion  fume 
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evacuation,  and  blowback  into  the  loading  area;  physical  space  requirements; 
and  internal  environmental  effects  (noise,  fumes,  heat). 


3.  INCREASED  COMPLEXITY 

The  addition  of  new  capabilities  to  an  existing  system  or  system  type 
will  generally  increase  complexity  of  the  system,  and  therefore  increase 
the  types  and  amount  of  testing  required  and  the  time  to  perform  these 
tests. 


E.g.,  the  addition  of  a  stabilized  turret,  a  missile  guidance  system, 
and  a  night  vision  device  to  a  tank  system  adds  electronic  complexities 
comparable  to  those  in  aircraft  avionics  suits.  Appropriate  electronic 
testing  in  addition  to  the  conventional  mechanical  testing  is  therefore 
required. 

4.  COMPONENT  INTERFACES 

Prior  to  assembly  in  a  prototype  system,  component  subsystems  should 
be  assembled  in  a  mockup  and  verified  for  physical  fit,  human  factors 
considerations,  interface  compatibility,  and  electrical  and  mechani¬ 
cal  compatibility. 

This  will  guarantee,  e.g.,  that  the  turret  can  be  traversed  with  all 
equipment  and  personnel  in  operational  configurations;  that  operation  of 
the  radio  transmitter  will  not  affect  the  fire  control  system,  and  opera¬ 
tion  of  the  fire  control  system  will  not  affect  the  night  vision  equipment; 
that  operation  of  the  gun  tube  will  not  physically  interfere  with  operation 
of  other  controls;  that  mutually  interdependent  systems  do,  in  fact,  operate 
as  planned;  and  that  operators  can  reach  and  use  all  controls  in  all  opera¬ 
tional  modes. 


5.  DETERMINATION  OF  TEST  CONDITIONS 

Test  conditions  during  validation  should  be  determined  by  the  primary 
objectives  of  that  test,  rather  than  by  more  general  considerations  of 
realism. 


Whenever  a  non-tactical,  non-operational  configuration  is  dictated  by 
test  requirements,  the  results  of  the  tests  should  not  be  challenged  by  the 
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fact  that  that  configuration  was  not  tactical  or  operational.  For  example, 
if,  in  the  development  of  a  truck  it  is  found  that  the  undercarriage  ground 
clearance  is  inadequate  for  operation  over  rough  terrain,*  it  may  be  desir¬ 
able,  for  a  variety  of  reasons,  to  operate  the  vehicle  only  over  smooth 
terrain  until  the  problems  of  ground  clearance  are  satisfactorily  solved. 

On  the  other  hand,  demonstration  and  acceptance  tests,  as  well  as  tests  in¬ 
tended  to  evaluate  performance  under  operational  conditions,  should  always 
be  conducted  under  conditions  as  close  to  those  anticipated  in  practice  as 
possible. 

6.  TEST  PLAN  DEVELOPMENT 

The  test  plan  developed  by  this  point  should  be  in  nearly  final  form. 

and  include  as  a  minimum: 


(a)  A  description  of  requirements,  Including: 

•  Performance 

•  Reliability 

•  Human  Factors 

•  Serviceability 

•  Maintenance 

(b)  A  test  definition  to  evaluate  the  requirements,  including: 

•  Performance;  speed,  brakes,  cooling,  steering, mounted  attach¬ 
ments,  missions,  gradeability,  stability,  swim,  mud,  altitude 
controllability. 

•  Reliability;  structural,  frame,  axle  housing,  turret,  artic¬ 
ulated  or  oscillation  joint,  attachments,  sheet  metal. 

•  Components;  engine,  transmission,  driveline,  hydraulic  system 
control  system,  suspension  system,  steering  system,  brakes, 
tires,  tracks. 

•  Human  Factors;  noise,  heat,  visibility,  control  location, 
ride,  communication,  function,  space. 


*If  examination  of  the  underside  of  a  truck  after  preliminary  drive-around 
exercises  reveals  indications  (scratches,  dirt  deposits)  of  contact  between 
the  ground  and  the  oil  sump,  then  full-fledged  cross-country  testing  (over 
rocks  in  particular)  should  not  be  attempted  until  a  skid  plate  or  other 
means  of  protection  has  been  installed. 
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•  Serviceability;  component  replacement  time,  component  repair 
time. 

•  Maintenance;  maintenance  points,  maintenance  periods. 

(c)  The  facilities  needed  to  make  evaluations,  including: 

•  Number  of  complete  vehicles;  number  of  subsystems  or  com¬ 
ponents,  spare  parts  support,  comparable  satisfactory  models 
for  comparison. 

•  Type  of  test  facilities  required;  proving  grounds  to  provide 
environmental  considerations,  instrumentation  capability  to 
measure  loads,  stresses,  strains,  temperatures,  pressures, 
etc.;  component  laboratories  for  fatigue  tests,  simulated 
tests,  accelerated  tests;  user  command  for  actual  mission 
evaluations. 

(d)  The  schedule  of  evaluations  and  facilities,  including  considera¬ 
tions  for  each  phase  of  development,  each  affected  organization, 
and  each  significant  development  item. 

(e)  The  reporting  procedure,  the  objective  of  which  is  to  communi¬ 
cate  tect  results  in  an  understandable  format  to  all  program 
echelons.  This  includes  problem  identification  to  the  design 
groups;  provision  of  data  on  stresses,  loads,  temperatures, 
etc.,  to  design  groups,  component  test  laboratories,  and  system 
test  laboratories;  and  result  summaries  as  reported  at  mile¬ 
stone  reviews  to  all  levels. 

(f)  The  Test  and  Evaluation  Guidelines,  which  include:  development 
of  an  endurance  test  course  where  the  user  helps  establish  the 
severity  (this  will  be  the  main  evaluation  of  reliability  and 
will  show  any  improvement  obtained);  continual  user  evaluation 
of  all  required  missions;  special  tests  which  must  be  estab¬ 
lished  to  validate  improvements;  identification  of  repeated 
problems  of  critical  items  and/or  failure  to  meet  mission  per¬ 
formance  requirements  which  call  for  immediate  evaluation  by 
the  program  manager  in  development  phase;  and  final  testing 
which  should  be  planned  on  first  production  units.  An  equi¬ 
valent  one  year’s  operation  on  the  endurance  test  course  may 
orevent  expensive  retrofit. 

(g)  A  further  refinement  of  the  cost  estimates  which  were  ini¬ 
tiated  during  the  conceptual  phase. 


7.  DEMONSTRATION  TESTS 

Demonstration  tests  should  show  satisfactory  meeting  of  success 
criteria  which  are  meaningful  in  terms  of  operational  usage. 
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In  designing  contractually  required  demonstration  tests,  upon  whose  out¬ 
come  may  depend  large  incentive  payments,  or  even  program  continuation,  it 
is  essential  to  specify  broader  success  criteria  than  simply  hit  or  miss  in 
a  single  given  scenario.  If  this  is  not  done,  the  entire  program  may  be 
skewed  to  meet  the  requirements  of  the  selected  scenario,  to  the  detriment  of 
exploring  the  entire  performance  envelope.  For  example,  the  hit  probability 
of  a  tank  gun  should  not  be  stated  solely  for  a  single  range  and  type  of 
ammunition  (e.g.,  0.9  for  APDS  round  at  1000  meters),  but  rather  over  a 
range  of  distances  and  types  of  ammunition  (e.g.,  0.5  at  2000  meters,  0.1 
at  3000  meters,  0.8  for  HEAT  at  1000  meters,  etc.)  With  too  much  weight 
attached  to  the  hit/miss  outcome,  non-tactical  hardware  may  be  retained 
beyond  the  early  stages  of  the  program  to  enhance  the  probability  of  success¬ 
ful  demonstration. 

Demonstrations  should  be  designed  to  measure  overall  performance,  with 
statistical  weighting  to  compensate  for  reduced  probabilities  of  success  at 
edge  values  of  condition  parameters. 

Results  of  tests  conducted  during  exploratory  development  and  which  most 
likely  have  been  conducted  on  brassboard,  breadboard,  or  modified  existing 
hardware  should  be  evaluated  with  special  attention  to  items  such  as: 

(a)  The  fabrication  of  the  vehicle  may  significantly  affect  the  per¬ 
formance  characteristics  so  that  the  suggested  proof  of  validation 
is  inconclusive. 

(b)  Scaling  laws  may  invalidate  the  findings  or  introduce  new 
technology  problems. 

(c)  The  laboratory  type  environment  in  which  the  vehicle  was  tested 
may  preclude  the  generation  of  data  needed  to  validate  that  the 
concept  and  technology  approach  will  be  applicable  to  an  opera¬ 
tional  environment. 

(d)  The  tests  may  not  include  vehicle  inputs  representative  of  those 
that  might  be  expected  in  an  operational  environment.  For  ex¬ 
ample,  the  ability  of  a  ground  vehicle  to  travel  on  side  hill 
slopes  must  be  evaluated  as  well  as  its  ability  to  climb  and 
descend  slopes.  In  addition  to  obvious  stability  problems  (if  a 
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side  hill  slope  is  too  steep,  the  vehicle  will  tip  over),  factors 
such  as  lubrication  (does  the  oil  level  drop  below  the  oil  pump 
pickup?)  and  fuel  delivery  (is  the  fuel  level  below  the  fuel 
pump  pickup?  Does  the  carburetor  float  level  change  excessively, 
either  starving  or  flooding  the  engine?)  must  also  be  considered. 

8.  RELIABILITY  TESTING 

Reliability  testing  should  be  performed  on  component  and  subsystem 

assemblies  prior  to  testing  of  the  complete  vehicle  system. 

Prior  to  full  system  testing  viable  component  and  subsystem  tests  should 
be  conducted.  Missile  test  adequacy  studies  show  that  almost  all  failures 
will  be  the  kind  that  cannot  be  detected  or  prevented  in  full  system  testing. 
While  this  may  not  be  as  true  for  ground  vehicles,  all  experience  indicates 
that  new  systems  will  exhibit  the  "new  system  syndrome"  and  that  by  far  and 
away  the  best  return  on  test  investment  will  come  from  applying  substantial 
attention  to  component  and  subsystem  level  test  effort.  Detecting  a  sub¬ 
system  or  component  failure  at  the  full  system  test  level  puts  them  at  the 
extremely  high  <  id  of  an  exponential  cost  curve. 

A  wheeled  or  tracked  weapon  system  (e.g.,  a  tank)  should  be  tested 
using  the  same  sequential  evaluations  as  any  other  weapon  system.  In  par¬ 
ticular,  components  (e.g.,  the  gun  tube,  the  turret  stabilization  system,  the 
fire  control  system)  should  be  bench  tested,  developed,  and  individually 
proven;  interfaces  should  be  examined;  and  system  operation  should  be  sim¬ 
ulated  prior  to  assembling  a  complete  prototype  system.  Individual  testing 
of  engine,  transmission,  and  suspension  would  be  followed  by  testing  of  a 
turretless  chassis.  For  example,  suspension  components  (tracks,  road  wheels, 
shock  absorbers,  torsion  bars,  sprockets,  idlers,  suspension  arms,  return 
rollers,  track  pads,  and  track  pins)  should  be  tested  individually  (e.g., 
testing  on  a  machine  which  will  put  complete  movement  cycles  on  a  shock 
absorber  at  a  high  rate  and  accumulate  operational  life  information  in  a  short 
period  of  time)  and  characteristics  matched  by  means  of  appropriate  simulators 
prior  to  assembly  on  a  test  platform.  Incorporation  on  a  prototype  should 
follow  satisfactory  test  platform  performance.  Simultaneously,  testing  of 
the  gun  tube,  stabilization  system,  and  night  sight  would  be  followed  by  test¬ 
ing  of  a  complete  turret,  perhaps  mounted  to  a  stripped  or  testbed  chassis. 
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A  preliminary  stress  survey  (using  stresscoat,  strain  gauges,  and  other 
appropriate  techniques)  should  be  made  of  all  loaded  parts  in  both  static 
and  dynamic  (including  fatigue)  configurations.  A  recheck  of  design  assump¬ 
tions  should  be  made  wherever  excessive  stress  levels  are  indicated,  and 
potential  problem  areas,  where  practical,  should  be  overloaded  to  failure 
to  determine  whether  safety  factors  are  adequate  for  anticipated  use. 
Similarly,  very  lightly  loaded  parts,  as  indicated  by  the  results  of  stress 
tests,  may  be  subject  to  re-examination  for  over-design.  Suspension  and 
driveline  components,  frames,  attachment  points  for  auxiliary  equipment  such 
as  blades  (as  well  as  the  blades  themselves) ,  and  ability  of  the  turret  and 
chassis  to  absorb  repeated  gun  tube  recoil,  are  normally  the  most  critical 
areas.  Basic  strength  evaluations  should  be  done  by  the  developer  prior  to 
submitting  the  prototype  system  for  test.  These  types  of  analysis  are  not 
normally  performed  during  service  system  test,  so  any  significant  structural 
modifications  subsequent  co  original  structural  tests  should  be  followed  by 
additional  checks  as  part  of  the  development  program.  Only  after  this  test¬ 
ing  is  finished  should  a  complete  tank  be  assembled  and  tested. 

9.  HUMAN  FACTORS 

In  evaluating  ground  vehicles,  human  factors  should  be  considered  at 

all  stages  starting  with  the  design  of  the  prototype. 

In  particular,  testing  should  be  conducted  to  determine  that 

(a)  The  smallest  operator  (fifth  percentile  or  other  appropriate  mea¬ 
sure)  and  the  largest  operator  (95th  percentile)  in  full  winter 
gear  can  utilize  any  passenger/operator  location  and  are  able  to 
reach  and  operate  all  controls  without  difficulty. 

(b)  Operators  and  passengers  are  able  to  communicate  as  needed,  and 
are  able  to  travel  in  a  reasonably  comfortable  fashion  (seats  pad¬ 
ded  and  adjustable) . 

(c)  Noise  levels  are  not  excessive  (e.g.,  95  pndb  maximum,  80  pndb 
sustained) . 

(d)  Vibration  and  shock  transmission  are  controlled. 

(e)  Temperature  environment  is  controlled  (e.g.,  to  between  30°  F 
and  90°  F  regardless  of  ambient). 

(f)  Limited  induction  of  dust  and  dirt  and  no  engine,  fuel,  or  exhaust 
fumes  are  allowed. 
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(g)  Protection  is  provided  £rom  wind,  rain,  and  other  undesirable 
conditions. 

(h)  Controls  are  so  designed  and  located  that  operation  is  straight¬ 
forward  and  easy,  and  possibilities  for  confusion  are  minimized. 

(i)  Tracked  vehicle  controls  are  as  similar  as  practicable  to  those  of 
wheeled  vehicles. 

(j)  Crew  members  are  capable  of  functioning  during  cross-country  oper¬ 
ation  of  combat  vehicles  at  maximum  speeds. 

(k)  Crew  comfort  and  operability  in  combat  vehicles,  space  use  optimi¬ 
zation,  and  turret/chassis  interface  are  considered  in  both  open 
and  buttoned-up  conditions. 

(l)  Instruments  and  controls  are  appropriately  lighted  for  night  oper¬ 
ations. 

(m)  Questions  of  safety,  appropriateness  of  man-machine  interfaces, 
and  the  number  and  skill  levels  of  the  personnel  required  are 
examined . 

(n)  The  numbers  of  personnel  required  are  validated  against  both  oper¬ 
ational  and  maintenance  requirements. 

Testing  early  versions  in  the  "human  acceptability  and  compatibility" 
environment  is  extremely  important.  This  will  also  help  to  validate  the 
manning  requirements. 

10.  TEST  PLAN  SCHEDULING 

Test  plan  scheduling  should  be  tied  to  event  milestones  rather  than  to 

the  calendar. 

In  evaluating  the  adequacy  of  the  scheduling  as  given  by  test  plans,  it 
is  important  that  milestones  be  tied  to  the  major  events  of  the  weapon  system 
(meeting  stated  requirements)  and  not  the  calendar.  As  a  result,  milestones 
should  be  flexible  with  respect  to  time.  The  acquisition  process  should  be 
based  on  the  achievement  of  major  milestones  and  sufficient  time  and  re¬ 
sources  allowed  between  these  milestones.  Flexibility  must  not  be  hampered 
by  the  contracting  mechanism.  A  system  of  effective  bottoms  up  review  at 
frequent  phases  of  the  program  should  be  established.  These  reviews  should 
be  objective,  in  depth,  and  provide  guidance  on  whether  or  not  to  commit  to 
the  next  phase.  It  is  inordinately  expensive  to  commit  to  a  test  program 
before  the  hardware  is  ready.  Component  problems  are  much  better  worked  at 
the  component  level  than  the  system  level.  Contractors  should  be  required 


to  demonstrate  successful  accomplishment  of  technical  milestones  before  pro¬ 
ceeding  to  the  next  phase  of  development.  For  example,  before  the  vehicle 
system  is  allowed  to  go  into  the  IOT&E  phase,  all  engineering  tests  should 
be  complete;  before  the  full-scale  production  phase  is  entered,  the  IOT&E  of 
the  total  vehicle  system  should  have  been  successfully  performed  by  the  using 
command;  prior  to  deployment  of  the  vehicle  system  to  the  user,  successful 
completion  of  the  acceptance  terting  of  che  initial  production  items  must 
have  occurred. 

In  evaluating  test  plans,  look  favorably  on  phasing  where  the  OT&E  is 
run  in  parallel  with  continued  DT&E.  A  problem  that  becomes  apparent  in  the 
operational  testing  can  often  be  evaluated  much  more  quickly  and  more  com¬ 
pletely  with  the  instrumented  DT&E  hardware. 

In  general,  DT  and  OT  plans  should  make  provisions  for  the  occurrence 
of  failures  and  in  particular  should  include  time  and  money  necessary  for 
investigating  test  failures  and  making  provisions  for  elimination  of  the 
cause  at  the  earliest  possible  date. 

11.  TEST  FAILURES 

The  T&E  schedule  should  be  sufficiently  flexible  to  accommodate  failures 

and  correction  of  problems  which  have  been  identified. 

T&E  schedules  must  provide  for  failures.  A  percentage  of  the  total 
tests  (sorties,  runs,  trials,  experiments)  should  be  allowed  for  retesting, 
over  and  above  the  minimum  number  required  to  successfully  complete  the  pro¬ 
gram.  This  percentage  must  be  related  to  the  probability  of  achieving 
success  as  opposed  to  failure.  On  a  "good"  program,  70  percent  of  all  test¬ 
ing  may  fall  into  the  category  of  testing  not  contained  in  the  original  pro¬ 
gram  test  plan;  on  a  program  with  problems,  it  may  go  higher.  An  early 
detection  scheme  for  top  government  and  contractor  management  should  be 
established  to  indicate  that  a  program  may  be  becoming  ill.  At  this  time 
there  may  be  a  good  possibility  of  recovery.  Some  of  the  indications  of 
trouble  during  the  engineering  development  phase  which  should  be  noted  by 
the  tester  are: 

(a)  Any  repetitive  failure  (e.g.,  repeated  failure  of  U-joints). 
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(b)  A  revision  of  schedule  or  incremental  funding  that  exceeds  the 
original  plan.  Predicted  downstream  recovery  may  not  have  a 
realistic  basis. 

(c)  Any  relaxation  of  basic  requirements  such  as  less  range,  lower 
performance,  etc.  (e.g.,  lower  maximum  speed,  higher  allowable 
[or  allowed]  transmission  and  differential  temperatures). 

Adequate  time  must  be  allowed  during  all  phases  of  testing  to:  (1) 
complete  all  necessary  testing  in  a  sequential  fashion  as  required;  and 
(2)  allow  the  developer  time  to  correct  problems  and  make  indicated  changes. 
Changes  or  modifications  to  the  system  require  some  or  all  of  the  testing 
to  be  repeated,  so  corrections,  once  identified,  should  be  expedited.  In 
particular,  changes  incorporated  as  a  result  of  pre-production  or  production 
vehicle  testing  should,  wherever  practical,  be  adequately  tested  prior  to 
commitment  to  full  scale  production  regardless  of  production  delays. 
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III.  FULL-SCALE  ENGINEERING  DEVELOPMENT  PHASE 

In  the  Full-Scale  Development  Phase,  the  engineering  development  and 
the  IOT&E  are  to  be  conducted.  Test  plans  prepared  during  the  previous 
Validation  Phase  should  be  refined  and  the  testing  will  be  conducted  in 
this  phase  aimed  at  demonstrating  that  a  substantial  production/deployment 
decision  is  warranted.  The  emphasis  during  this  phase  should  be  placed  on 
the  evaluation  of  the  vehicle  against  the  test  plan  requirements  including: 

•  Performance 

•  Reliability 

•  Human  Factors 

•  Serviceability 

•  Maintenance 

Validation  and  full  scale  development  phase  checklist  items  are  approp¬ 
riate  guidance  for  actions  before  the  production  decision  DSARC.  Again,  guide¬ 
lines  from  the  previous  phases  may  continue  to  apply  and  should  be  reviewed. 

The  full  scale  development  checklist  includes: 

1.  Planning  the  Operational  Test 

2.  Pilot  and  Dry  Run  Tests 

2.  Comparison  Testing 

4.  Simulations 

5.  Environmental  Testing 

6.  System  Vulnerability 

7.  Design  Criteria  Verification 

8.  System  Critical  Speeds 

9.  Electromagnetic  Testing 

10.  System  Strength  Testing 

11.  Component  Compatability 

12.  Human  Interface 

13.  Serviceability  Testing 

14.  Experienced  User  Critique 

15.  Troubleshooting  During  Tests 


Preceding  page  blank  2? 


1. 


PLANNING  THE  OPERATIONAL  TEST 


Operational  testing  should  be  cost  effective  and  provide  meaningful 
results. 


Operational  testing  is  essential,  but  it  is  also  expensive  and  time 
consuming.  Be  sure  that  the  test  plans  are  well  thought  out  so  that  the 
value  received  is  worth  the  effort  expended  and  at  least  cost  equivalent  to 
the  not-delivered  systems.  Think  in  terms  of: 

(a)  Involving  operational  groups  in  test  planning  and  in  establishing 
measures  of  effectiveness,  so  that  the  outcome  of  the  vehicle 
system  tests  will  be  accepted  as  being  operationally  significant. 

(b)  Determining  whether  the  scope  of  the  planned  tests  will  provide 
sufficient  data  on  the  performance  of  the  vehicle  system  to 
justify  any  change  at  all  in  the  eyes  of  potential  users. 

(c)  Comparing  the  scope  of  proposed  tests  against  checklists  of 
vehicle  system  issues  frequently  raised  at  major  decision  mile¬ 
stones,  to  assure  that  the  data  needed  for  such  decisions  will  be 
forthcoming  to  the  extent  this  is  possible  from  testing  alone. 

(d)  Recognizing  in  the  formulation  of  test  plans  that  major  system 
decisions  are  judgments  based  on  a  wide  range  of  qualitative 
considerations,  as  well  as  on  statistical  compilations,  and  that 
the  outcome  and  limitations  of  operational  tests  must  be  compre¬ 
hensive  and  meaningful  to  the  decision  makers  as  well  as  to  the 
testing  community.  For  example,  if  a  major  design  consideration 
is  a  swim  capability,  then  the  results  of  OT&E  should  reflect 
vehicle  performance  in  river  crossings.  Data  should  include 
depths,  stream  velocity,  entrance  and  departure  bank  slopes,  etc. 


2.  PILOT  AND  DRY  RUN  TESTS 

A  scheduled  series  of  tests  should  be  preceded  by  a  dry  run  which 
verifies  that  the  desired  data  will  be  obtained. 


Before  tests  for  demonstration  of  operational  suitability  and  effec¬ 
tiveness  are  conducted,  a  pilot  test  should  be  held  with  the  primary  purpose 
of  shaking  down  the  test  plan,  the  instrumentation  concept  and  the  data 
analysis  plan.  A  secondary,  but  vital  purpose  should  be  to  provide  final 
training  for  the  test  participants.  In  general,  a  pilot  test  should  be 
conducted  sufficiently  prior  to  the  OT&E  so  that  ample  time  is  available 
to  make  the  necessary  changes  to  the  OT&E  as  dictated  by  the  results  of 
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the  pilot  test.  A  full-throttle  full-load  dry  run  test  which  indicates 
that  the  engine  will  overheat  at  ambient  temperatures  above  110°  F  indi¬ 
cates  that  testing  at  125°  F  ambient  should  not  be  attempted. 

Dry  runs  should  be  conducted  for  each  new  phase  of  testing.  For  exam¬ 
ple,  simulation  and  other  laboratory  or  ground  testing  could  be  conducted 
to  predict  the  specific  test  outcome.  The  DT/OT  tests  should  then  be  run 
to  verify  the  test  objectives.  Evaluation  of  the  simulation  vis-a-vis  the 
actual  test  results  would  help  to  refine  the  understanding  of  the  system. 

3.  COMPARISON  TESTING 

The  test  program  should  include  a  detailed  comparison  of  the  charac¬ 
teristics  of  a  new  vehicle  system  with  those  of  existing  systems, 
alternate  vehicle  system  concepts  (if  applicable),  and  those  of  any 
system(s)  being  replaced. 

A  portion  of  the  test  program  should  be  devoted  to  comparison  testing 
(side-by-side  when  practical)  between  the  new  vehicle  and  the  actual  ob¬ 
served  characteristics  of  the  vehicle(s)  being  replaced.  When  significant 
degradation  in  various  measures  of  performance  are  found  (e.g.,  slower  ac¬ 
celeration,  lower  top  speed,  poorer  fuel  economy,  poorer  reliability,  high¬ 
er  maintenance,  increased  operating  costs,  reduced  payload  weight,  reduced 
payload  volume,  loss  of  mobility,  loss  of  swimming  capability,  harder  to 
operate),  the  degradation  should  be  emphasized  in  the  test  report  as  well 
as  the  probable  cause(s),  (e.g.,  reduced  payload  volume  due  to  smaller  car¬ 
go  bed);  poorer  fuel  economy  due  to  less  efficient  engine  design  (undesir¬ 
able)  or  revised  gearing  to  improve  acceleration  (a  conscious  trade-off 
decision).  In  many  cases  (e.g.,  fuel  economy,  cross-country  mobility), 
there  are  multiple  interacting  factors  to  be  considered,  all  of  which  should 
be  identified.  Contradictory  results  may  be  obtained  (e.g.,  vehicle  has 
excessive  power  resulting  in  undesirably  high  top  speed  on  paved  roads,  but 
is  underpowered  for  high  mobility  cross  country),  but  the  purpose  of  the 
testing  is  to  identify,  not  cure,  any  potential  problem  areas. 
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SIMULATIONS 


Simulation  techniques  and  equipment  should  be  utilized  to  enhance 

data  collection. 

Creation  of  histograms  for  each  test  course  provides  a  record  of  con¬ 
ditions  experienced  by  the  vehicle  during  testing.  Use  of  a  chassis  dyna¬ 
mometer  can  produce  additional  driveline  endurance  testing  with  more  com¬ 
plete  instrumentation  coverage.  Programmed  engine  dynamometers  can  provide 
accelerated  engine  and  transmission  endurance  testing,  and  simulate  extreme 
conditions  (e.g.,  full  throttle,  full  load,  high  coolant  and  lubricant 
temperatures)  for  extended  periods  of  time*  rather  than  depending  on  the 
length  of  the  available  grade  and  a  high  ambient  temperature.  Test  course 
failures  attributable  to  loaded  component  failures  should  be  duplicated  on 
the  simulator  to  verify  causes.  Simulators  are  also  valuable  in  estimating 
limits  to  the  performance  envelope  (e.g.,  engine  coolant  temperatures  reach 
unacceptable  levels  at  a  still  air  ambient  of  127°  F) .  Note  that  successful 
completion  of  a  20,000-mile  endurance  test  (or  an  accelerated  test  to  a 
shorter  distance)  on  a  simulator  does  not  eliminate  the  need  for  field 
testing,  but  may  reduce  the  number  of  test  vehicles  required. 

5.  ENVIRONMENTAL  TESTING 

Ground  vehicles  should  be  tested  jn  environmental  conditions  and  situ¬ 
ations  comparable  to  those  in  which  they  will  be  expected  to  perform.** 

These  should  include  tropical,  desert,  mud  and  jungle,  cross  country 
operations,  including  mountains,  and  elevations  from  sea  level  to  in  excess 
of  10,000  feet.  Areas  of  particular  concern  should  include: 

(a)  Mobility  (Does  the  vehicle  get  stuck?) 

(b)  Engine  Starting  (Will  it  start  at  -40°  F?  +20°  F?) 


•/( 

Extended  periods  of  time  and  severity  of  conditions  must  be  specifically 
defined  for  each  system  in  terms  of  its  intended  application. 

A 

Also,  by  implication,  testing  should  not  be  required  in  environment"  in 
which  the  vehicle  will  not  be.  expected  to  perform.  A  snowmobile  wouli  not 
be  subjected  to  desert  testing. 
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(c)  Engine  Performance  (Will  it  pull  rated  loads  up  representative 
grades  at  high  elevations?) 

(d)  Cooling  (Does  the  engine  coolant  overheat  under  full-load,  full- 
throttle  conditions?  Do  engine,  transmission,  or  differential 
lubricants  overheat  under  heavy  load  conditions?  Are  there  vapor 
lock  problems?) 

(e)  Wear  (Do  brake  linings  wear  very  rapidly  when  operated  in  muddy 
conditions?  Is  track  wear  excessive  on  paved  roads?) 

(f)  Crew  Comfort  (Is  operator  performance  impaired  because  of  exces¬ 
sively  high  or  low  internal  temperatures,  etc?) 

(g)  Maintenance  Requirements  (Does  air  cleaner  require  excessive  ser¬ 
vice  ‘n  dusty  environments?  Does  radiator  collect  dust  and  dirt 
depo  ?  Are  components  too  hot  [or  cold]  to  service,  especially 
during  field  failures?  Is  servicing  in  rain,  snow,  dust,  wind, 
high  humidity  undesirable?) 

(h)  Reliability  (Do  controls  such  as  throttle  and  brake  freeze  up 
when  vehicle  is  operated  in  slushy  snow?  Do  switches  corrode 

and  become  inoperable  in  high  humidity  or  salt  spray  environments? 
Do  wheel  bearings  fail  prematurely  when  operated  in  very  cold 
ambient  temperatures?  Do  optical  components  fog  up  in  high  tem¬ 
perature,  high  humidity,  environments?) 


6.  SYSTEM  VULNERABILITY 

For  combat  vehicles,  some  estimate  of  vulnerability  to  battle  damage 
should  be  made. 

If  the  operator  is  in  an  armor-protected  position,  he  is  protected 
from  rifle  fire  and  shell  fragments,  but  may  be  vulnerable  to  a  land  mine 
explosion.  The  relative  vulnerability  of  other  "soft"  components  (radia¬ 
tor,  fuel  tank,  fuel  lines,  coolant  hoses,  some  engine  components,  tires) 
is  a  function  of  both  the  inherent  protection  of  the  item  (a  fuel  line 
running  inside  a  boxed  frame  rail  is  protected  from  many  hazards;  the  same 
fuel  line  running  along  the  bottom  of  the  same  frame  rail  is  subject  to 
damage  by  large  rocks  as  well  as  a  complete  assortment  of  ordnance)  and 
any  special  measures  which  are  taken  (armor  protection;  fuel  cells  inside 
fuel  tanks;  foam-filled  tires;  bullet  and  shell  fragment  deflecting  radia¬ 
tor  grilles).  Tracks  and  suspension  systems  are  vulnerable  to  damage  that 
can  immobilize  tracked  vehicles.  Fire  controls,  particularly  vision  de¬ 
vices,  are  subject  to  damage  that  can  destroy  the  effectiveness  of  combat 
vehicle  weapons. 

31 


7. 


DESIGN  CRITERIA  VERIFICATION 


Subsystem  design  criteria  should  be  compared  with  actual  characterls- 
tics. 


E.g.,  a  track  may  be  designed  to  operate  at  a  ground  pressure  of  4  psi, 
but  if  the  actual  tank  weight  is  25  percent  higher  than  the  original  target 
weight,  the  track  may  be  operating  at  a  25  percent  increase  in  ground  pres¬ 
sure,  with  corresponding  effects  on  mobility.  Similarly,  a  turret  stabili¬ 
zation  system  may  be  designed  for  a  specific  turret  moment  of  inertia; 
changes  in  the  turret  weight  (and  moment  of  inertia)  may  severely  degrade 
the  stabilization  system  performance. 

8.  SYSTEM  CRITICAL  SPEEDS 

Critical  speeds  should  be  determined  for  all  ground  vehicles. 

Critical  speeds  include,  e.g.,  those  at  which  resonant  vibration  fre¬ 
quencies  are  encountered.  If  tank  firo  control  system  optics  encounter 
a  resonance  at  an  operational  road  speed,  performance  would  be  significant¬ 
ly  degraded  if  either  the  system  was  not  mounted  using  shock  and  vibration 
isolation  techniques,  or  the  tank  was  not  modified  as  necessary  to  elimi¬ 
nate  the  resonance. 

9.  ELECTROMAGNETIC  TESTING 

Vehicle  testing  should  include  electromagnetic  testing. 

E.g.,  for  a  tank,  this  includes  evaluation  of:  the  electromagnetic 
signature  of  the  tank  as  a  function  of  which  systems  are  operating;*  inter¬ 
nal  system  compatibility  (e.g.,  does  the  radio  transmitter  affect  the  tur¬ 
ret  stabilization  system  at  certain  broadcast  frequencies?);  effects  of 
external  electromagnetic  radiation  (do  fuzes  detonate  when  a  range-finding 
radar  is  trained  on  the  tank?);  and  compatibility  with  other  current  and 


*This  includes  optical  and  IR  signatures,  both  day  and  night,  with/without 
engine  operating,  before/after  gun  firing,  etc. 
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proposed  electromagnetic  generating  systems  (tanks,  aircraft;  radars,  etc.) 
during  the  contemplated  operational  time  frame. 

10.  SYSTEM  STRENGTH  TESTING 

In  evaluating  ground  vehicles,  early  testing  should  verify  intrinsic 

strength. 

This  implies  operation  with  maximum  anticipated  loading,  including 
trailed  loads  at  maximum  speeds  and  over  worst  case  grades,  secondary  roads, 
and  cross-country  conditions  for  which  the  vehicle  was  developed  or  procur¬ 
ed.  This  test  is  Intended  to  identify  deficient  areas  of  design,  not  to 
break  the  machinery.  Areas  which  should  be  examined  for  damage  or  incipient 
failure  include:  frame,  tires,  wheels,  tracks,  springs,  shock  absorbers, 
torsion  bars,  wheel  bearings,  suspension  bushings,  tie  rods,  drag  links, 
king  pins  (or  ball  joints  or  other  equivalent  suspension  components),  U- 
joints,  drive  shafts,  axle  shafts,  propeller  shafts,  transfer  cases,  trans¬ 
missions,  differentials,  axle  housings,  engines,  cooling  systems,  including 
radiators  and  hoses  (both  cooling  capability  and  structural  integrity  are 
important),  auxiliary  drives,  steering  gear,  tracks,  sprockets,  road  wheels, 
and  any  other  components  which  may  be  subjected  to  loading  during  severe 
operating  conditions.*  Any  deficient  components  should  be  strengthened  as 
needed  prior  to  performing  any  further  testing,  which  should  begin  with  a 
check  test  of  the  entire  system  to  verify  the  modified  components.  Testing 
may  need  to  be  repeated  in  the  event  of  redesign  or  modification  of  a 
component  for  any  reason. 

11.  COMPONENT  COMPATIBILITY 

Component  compatibility  should  be  checked  through  the  duration  of  the 

test  sequence. 

Suspension  components  should  be  tested  to  verify,  for  example: 

•  That  shock  absorbers  and  springs  are  matched  to  produce  the  desir¬ 
ed  transient  response  to  dynamic  inputs. 

*A  severe  condition  is  defined  in  terms  of  a  baseline  set  by  an  existing 
similar  vehicle  configuration. 
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•  That  wheel  and  suspension  bearings  are  an  adequate  size  for  the 
loads  imposed  during  testing  and 

•  That  tracks  are  compatible  with  suspension  components. 

The  driveline — engine,  transmission,  axles,  etc. — should  be  a  bal¬ 
anced  design,  and  the  interfaces  with  other  portions  of  the  vehicle  (e.g., 
motor  and  transmission  mounts,  spring  hangers,  shock  absorber  attachment 
points)  should  be  checked  for  premature  failure.  Vibration  testing  on 
vibration  fixtures  to  determine  resonant  frequencies  of  items  such  as  fuel 
tank  support  brackets  can  be  used  to  predict  potential  areas  of  fatigue 
failure. 

Reverification  of  compatibility  may  be  required  in  the  event  of  any 
significant  change  in  components  or  subsystems.  The  magnitude  of  this 
testing  should  be  compatible  with  the  magnitude  of  the  change.  If  an  engine 
is  changed,  almost  the  entire  test  program  may  require  repetition,  while  if 
a  rear  view  mirror  or  turn  signal  were  changed  most  portions  of  the  test  would 
not  need  repetition  (note  that  arctic  testing  might  reveal  that  the  mirror 
frame  contracts  and  cracks  the  mirror,  while  desert  testing  may  reveal  that 
insulation  in  the  turn  signal  softens  under  extreme  heat,  resulting  in  a 
short  circuit.  Simulation  of  these  conditions  [i.e.,  in  a  freezer  and  an 
oven]  may  reveal  these  problems  without  the  need  tor  a  full  environmental 
test.  Note  also  that  actual  desert  maximum  temperatures  [e.g.,  125°F  ambient 
plus  solar  heating  loads],  rather  than  extrapolation  from  lower  temperature 
levels,  are  required  to  demonstrate  this  type  of  failure). 

12.  HUMAN  INTERFACE 

Critiques  of  good  and  bad  features  of  the  vehicle  should  be  made 

early  in  the  prototype  stage,  while  adequate  time  remains  to  make 

any  indicated  changes. 

Note  this  includes  features  which  may  not  be  spelled  out  specifically 
in  the  test  plan;  e.g.,  that  an  annoying  resonance  occurs  at  typical  convoy 
speed,  that  engine  surge  is  annoying,  that  headlight  and  windshield  wiper 
switches  can  be  confused,  that  the  clutch  chatters  when  engaged  from  rest 
on  an  uphill  location,  that  the  seat  is  extremely  comfortable,  that  some 
controls  cannot  be  reached  if  seat  belts  are  used,  that  the  external  rear 


V 


34 


view  mirror  vibrates  to  an  unusable  extent  and  reflects  the  headlights  of 
following  vehicles  at  night  with  excessive  glare,  that  rear  vision  while 
backing  up  is  outstanding,  that  a  fiberglass  battery  cover  makes  a  conven¬ 
ient  step  but  is  not  strong  enough  to  sustain  a  man's  weight,  that  the  wind¬ 
shield  wiper  arm  contacts  a  blackout  light  wire  when  operating  and  will 
eventually  abrade  the  insulation,  and  that  the  hooks  on  the  chains  which 
hold  the  tailgate  in  the  down  position  are  inadequate  to  support  a  heavy 
load  or  a  man  standing  on  the  tailgate.  The  need  for,  e.g.,  a  step  to  aid 
climbing  into  the  cab,  would  also  be  expressed  at  this  time.  Fire  control 
layout  in  combat  vehicles  should  be  considered  with  regard  for  rapid  opera¬ 
tion  under  conditions  of  stress. 

13.  SERVICEABILITY  TESTING 

Ground  vehicles  should  be  tested  and  evaluated  to  determine  the  rela¬ 
tive  ease  of  serviceability,  particularly  with  high  frequency  opera¬ 
tions. 

Test  items  should  include  answers  to  the  following  questions:  Is  it 
easy  to  change  spark  plugs  or  to  clean  and  service  injector  pumps?  Are 
there  access  points  for  diagnostic  equipment?  Can  wearing  parts,  such  as 
brake  linings,  be  examined  easily?  Can  oil  level,  coolant  level,  battery 
water  level,  and  other  fluid  levels  be  checked  easily?  Can  wheel  align¬ 
ment  be  checked  and  adjusted  using  available  servicing  equipment?  Can 
starter,  alternator,  carburetor  or  injector  system,  distributor  (if 
applicable),  oil  filter,  and  other  engine  accessories  be  reached,  examined, 
and  easily  removed  for  servicing  or  replacement?  Can  all  lubrication 
points  be  located  and  serviced  easily?  Are  critical  components  (oil  pan, 
coolant  hoses,  brake  lines)  protected  from  contact  with  rocks,  etc.,  yet 
easily  inspected?  Can  track  tension  be  adjusted  easily  by  crew  members? 

Can  a  thrown  track  be  remounted  under  field  conditions? 

14.  EXPERIENCED  USER  CRITIQUE 

Ground  vehicle  user  opinions  should  be  obtained  early  in  the  develop¬ 
ment  program. 
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Test  drivers  with  extensive  experience  frequently  miss  problems  which 
most  users  identify  fairly  rapidly  (e.g.,  a  user  who  has  been  driving  gas¬ 
oline-powered  automobiles  with  automatic  transmissions  may  have  initial 
difficulties  starting  and  driving  a  standard  transmission  truck  powered  by 
a  diesel  engine.  An  experienced  tester  would  make  the  necessary  compensa¬ 
tions  for  differences  among  various  vehicles).  Users  should  also  enter 
into  the  maintenance  evaluation  (for  example,  the  most  efficient  way  to 
change  starters  may  be  to  pull  the  engine;  the  user,  who  does  not  have  an 
engine  hoist  in  the  field,  may  find  that  a  revised  frame  cross  member  pro¬ 
hibits  starter  removal  according  to  the  scheme  described  in  the  maintenance 
manual).  User  maintenance  must  be  capable  of  being  performed  on  combat 
vehicles  under  all  likely  field  conditions. 

15.  TROUBLESHOOTING  DURING  TESTS 

Provisions  should  be  made  to  identify  subsystem  failure  causes. 

Subsystems  may  exhibit  failures  during  testing.  Adequate  provisions 
should  be  made  to  permit  troubleshooting  and  identification  of  defective 
components  and  inadequate  design.  E.g.,  a  turret  stabilization  system 
which  includes  a  set  of  black  boxes  fails  to  operate.  If  test  equipment 
and  troubleshooting  techniques  are  not  provided,  the  tester  has  no  way  to 
distinguish  among  a  fuse  failure,  a  defective  electrical  component  (capaci¬ 
tor,  resistor,  transistor,  integrated  circuit),  and  a  defective  design  or 
assembly. 
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j  IV.  SUBSTANTIAL  PRODUCTION/DEPLOYMENT  PHASE 
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The  purpose  of  this  phase  is  to  verify  production  design  utilizing  early 
production  vehicles.  The  lead  time  after  the  production  decision  and  before 
full  production  is  rarely  less  than  2  years,  and  may  be  4  years  or  more;  con¬ 
sequently,  the  detailed  planning  for  OT&E  can  probably  wait-  until  after  the 
production  decision.  However,  prior  to  that  time  the  basic  OT&E  plans  should 
have  been  made  for  manning  assignments,  personnel  training,  hardware  and  soft¬ 
ware  requirements  and  facilities. 

Some  of  the  full-scale  development  phase  testing  will  be  continued  into 
this  phase,  and  appropriate  checklist  items  would  also  carry  over.  The 
checklist  includes: 

1.  Performance  and  Reliability  Testing 

2.  Lead-the-Fleet  Testing 

3.  User  Evaluation 
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1.  PERFORMANCE  AND  RELIABILITY  TESTING 

The  production  first-article  testing  should  verify  the  performance  of 
the  vehicle  system  and  determine  the  degradation,  failure  modes,  and 
failure  rates. 

The  production,  first-article  testing  and  evaluation  should  be  design¬ 
ed  and  conducted  to  confirm  the  adequacy  of  the  vehicle  system  to  meet 
specified  performance  requirements,  to  evaluate  variation  of  critical  perfor¬ 
mance  parameters  with  time,  and  to  determine  failure  modes  and  rates.  These 
development  tests,  usually  conducted  first  by  the  contractor  and  then  by  the 
service,  should  be  functional  at  the  subsystem  level  but  should  progress  to 
the  fully  integrated  system  test  leading  to  a  service  acceptance  test.  This 
activity  should  be  followed  by  FOT&E. 

In  practice,  the  larger  sample  size  available  for  distribution  at  the 
battalion  level  may  reveal  previously  unidentified  problems,  and  modifica¬ 
tions  made  to  enhance  produceability  may  affect  performance. 

Ideally,  operational  testing  of  production  items  should  be  primarily 
an  evaluation  of  reliability/maintainability/availability.  Problems  should 
also  be  anticipated  in  the  fields  of  quality  control  and  assembly  procedure. 
The  basic  durability  and  performance  should  have  been  established  previously, 
and  inherent  component  and  subsystem  failures  should  not  be  encountered.  If 
any  of  the  latter  types  of  problems  are  identified,  there  should  be  provi¬ 
sion  for  stopping  or  modifying  production  until  the  causes  are  identified  and 
cured . 

2.  LEAD-THE-FLEET  TESTING 

At  least  one  production  prototype  or  initial  production  model  vehicle 
should  be  allocated  to  intensive  testing  so  as  to  accumulate  very  high 
operating  time  in  a  short  period. 

The  testing  should  be  performed  over  the  same  test  course  used  to  define 
the  reliability  characteristics  of  the  vehicle.  It  places  the  vehicle  under 
the  various  stress  and  strain  conditions  to  be  expected  of  it  during  typical 
combat  conditions  (if  it  is  a  combat  type  vehicle),  or  under  normal  operat¬ 
ing  conditions  (if  it  is  a  non-combat  type  vehicle.)  This  testing  is  not  to 
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replace  or  reduce  the  need  for  a  standard  endurance  test  but  is  an  operation¬ 
al  usage  test  of  the  total  weapon  system  under  normal  conditions  maintained 
at  an  accelerated  pace  so  that  problems  can  be  identified  and  resolved  very 
early  in  the  production  of  the  vehicle. 

The  lead-the-f leet  concept  is  meaningful  for  a  tank  only  when  all  sys¬ 
tems  are  equally  exercised  at  operational  levels.  Accumulation  of  chassis 
miles  may  be  meaningful  from  the  standpoint  of  power  train  evaluation,  but 
it  is  not  adequate  for  weapon  systems  evaluation.  Gun  tests  (including 
accuracy),  stabilization  system  performance,  fire  control  system  performance, 
ranging  system  performance,  night  vision  system  performance,  and  communica¬ 
tions  system  performance,  are  all  significant  as  well. 

3.  USER  EVALUATION 

User-reported  shortcomings  should  be  followed  up  to  determine  problem 

areas  requiring  correction. 

After  the  new  vehicle  system  is  placed  with  the  user,  there  are  always 
problems.  These  problems  ire  reported  through  materiel  channels,  e.g.,  as 
unsatisfactory  equipment  reports  (UERs)  or  deficiency  reports  (DRs) .  To 
plan  appropriate  follow-on  OT&E,  these  unsatisfactory  conditions  must  be 
closely  investigated  so  that  corrections  can  be  candidates  for  more  complete 
evaluation  in  FOT,  with  redesign  requirements  to  be  determined.  Identifica¬ 
tion  of  problem  areas  at  this  stage  should  not  be  considered  to  be  a  mis¬ 
application  of  equipment. 
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